Skip to content

fix: make Node microbenches more stable#376

Open
cheb0 wants to merge 4 commits intomainfrom
0-fix-agg-benchmarks-variance
Open

fix: make Node microbenches more stable#376
cheb0 wants to merge 4 commits intomainfrom
0-fix-agg-benchmarks-variance

Conversation

@cheb0
Copy link
Member

@cheb0 cheb0 commented Mar 6, 2026

Description

Use const seed and call ResetTimer. They false report degradation too often, but it's possible to slow down on them actually.


  • I have read and followed all requirements in CONTRIBUTING.md;
  • I used LLM/AI assistance to make this pull request;

@codecov-commenter
Copy link

codecov-commenter commented Mar 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.59%. Comparing base (03387c3) to head (28e33ef).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #376      +/-   ##
==========================================
+ Coverage   71.40%   71.59%   +0.19%     
==========================================
  Files         206      206              
  Lines       15057    15057              
==========================================
+ Hits        10751    10780      +29     
+ Misses       3523     3491      -32     
- Partials      783      786       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2026

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=1000-4 ad0347 96051b
4737.00 ns/op 5498.00 ns/op 1.16 🔴
AggDeep/size=10000-4 ad0347 96051b
47818.00 ns/op 56106.00 ns/op 1.17 🔴
AggWide/size=1000-4 ad0347 96051b
4780.00 ns/op 5920.00 ns/op 1.24 🔴
FindSequence_Random/medium-4 ad0347 96051b
12292.95 MB/s 11074.57 MB/s 0.90 🔴

@github-actions
Copy link
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=10000-4 03387c 7bc28e
45145.00 ns/op 55798.00 ns/op 1.24 🔴
And/size=1000-4 03387c 7bc28e
4.29 ns/op 4.82 ns/op 1.12 🔴

@eguguchkin eguguchkin added this to the v0.69.0 milestone Mar 16, 2026
@eguguchkin eguguchkin requested review from dkharms, eguguchkin and forshev and removed request for eguguchkin March 16, 2026 10:09
n := NewOr(newNodeStaticSize(s), newNodeStaticSize(s), false)
n := NewOr(newNodeStaticSize(r, s), newNodeStaticSize(r, s), false)

b.ResetTimer()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Loop itself takes care of resetting the timer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@github-actions
Copy link
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=10000-4 03387c 349c14
45145.00 ns/op 56829.00 ns/op 1.26 🔴
AggWide/size=1000-4 03387c 349c14
5522.00 ns/op 6196.00 ns/op 1.12 🔴
FindSequence_Random/small-4 03387c 349c14
6427.67 MB/s 5371.19 MB/s 0.84 🔴
40.85 ns/op 47.66 ns/op 1.17 🔴
MutexListAppend-4 03387c 349c14
187.29 MB/s 169.75 MB/s 0.91 🔴
ProcessDocuments-4 03387c 349c14
29061.00 B/op 32403.00 B/op 1.11 🔴
101.50 MB/s 92.16 MB/s 0.91 🔴

@github-actions
Copy link
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=10000-4 03387c 8bd3d8
45145.00 ns/op 57069.00 ns/op 1.26 🔴
AggWide/size=1000-4 03387c 8bd3d8
5522.00 ns/op 6223.00 ns/op 1.13 🔴
FindSequence_Random/large-4 03387c 8bd3d8
14232.04 MB/s 12657.88 MB/s 0.89 🔴
FindSequence_Random/medium-4 03387c 8bd3d8
11010.51 MB/s 9913.98 MB/s 0.90 🔴
FindSequence_Random/small-4 03387c 8bd3d8
6427.67 MB/s 5421.49 MB/s 0.84 🔴
40.85 ns/op 47.22 ns/op 1.16 🔴
ProcessDocuments-4 03387c 8bd3d8
29061.00 B/op 32768.00 B/op 1.13 🔴
101.50 MB/s 91.66 MB/s 0.90 🔴

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants